An exploratory study of dynamic foot shape measurements with 4D scanning system

Accurate and reliable foot measurements at different stances offer comprehensive geometrical information on foot, thus enabling a more comfortable insole/footwear for practical use and daily activities. However, there lacks investigations on continuous deformation of foot shape during the roll-over process. This study analyses the foot deformation of 19 female diabetic patients during half weight bearing standing and self-selected walking speed by using a novel 4D foot scanning system. The scanning system has good repeatability and accuracy in both static and dynamic scanning situations. Point cloud registration for scanned image reorientation and algorithms to automatically extract foot measurements is developed. During the foot roll-over process, maximum deformation of length and girth dimensions are found at first toe contact. Width dimensions have maximum deformation at heel take off. The findings provide a new understanding of foot shape changes in dynamic situations, thus providing an optimal solution for foot comfort, function and protection.


Scientific Reports
| (2023) 13:8628 | https://doi.org/10.1038/s41598-023-35822-0 www.nature.com/scientificreports/ process is time-consuming and prone to errors. The use of different scanning systems has however resulted in a large volume of point clouds, so algorithms have been developed for point cloud registration to increase the data processing efficiency and accuracy in previous studies [18][19][20][21][22] . Point cloud registration has been widely used in many computer vision tasks, like 3D reconstruction 23,24 , object detection 25 , pose and motion estimation [26][27][28] , autonomous driving 29,30 , robotic manipulation 31 , medical imaging [32][33][34] , etc. It is an important step to merge multiple data sets into a globally unified model, or map an unknown data set onto a known data set to identify its features or estimate its pose. Besl and Mckay 20 described a well-known method for point cloud registration based on the iterative closest point (ICP) algorithm. This algorithm repeatedly selects the corresponding pairs of points between the target and source point clouds, then calculates the optimal translation, rotation, and scaling up or down, and finds the transformation parameters between the corresponding pairs of points by using the least squares method. This process is repeated until there is an optimal match between the two sets of point clouds to meet various metric criteria, like the root-mean-square error (RSME) 35 . Another robust probabilistic method for point cloud registration based on a coherent point drift (CPD) algorithm was introduced by Myronenko and Song 22,36 . The CPD algorithm is used to consider the alignment of two point sets as a probability density estimation problem 22,37 , where the probability value of 1/0 represents the true/false correspondence respectively, and a larger probability refers to greater certainty of the correspondence. Given that there are two point sets M and S, they represent the distribution of M as a Gaussian Mixture Model (GMM), where the corresponding likelihood function peaks when M is fully aligned with S. In this study, the CPD algorithm is used for registering foot images to a standard coordinate for measurement purposes, and then the key foot measurements related to the shoe/insole design are extracted automatically.
Therefore, the primary aims of this study are: (1) To validate the repeatability and accuracy of the 3dMD foot scanning system, (2) To explore a suitable approach for foot image reorientation based on point cloud registration so that key foot measurements can be automatically extracted, and (3) To analyse foot roll-over shapes for patients with diabetes.

Methodology
Dynamic foot scanning system. The 3dMD foot scanning system (3dMD LLC, AG, USA) is also known as a 4D foot scanning system, which has the additional dimension of time as opposed to the three dimensions of 3D scanning systems. The system contains 10 machine vision cameras, including dedicated cameras (green circle labelled in Fig. 1) that capture the plantar of the foot with projection through glass or customised speckle patterns. The speckle patterns enable the accurate reconstruction of the foot shape when in contact with the glass (length * width: 42 * 19 cm). They provide a 360-degree geometric shape of the entire foot for both the surface of the dorsal and plantar of the foot with articulation and movement during locomotion, and a progressive sequence of 3D foot images (40 fps) over a period of time can be recorded, which enables the operator to select the optimal frame in real time for immediate analysis or render the entire image sequence for further evaluation and use. The system rendering engine incorporates information from all camera viewpoints per frame so there is no manual stitching required. The accuracy of the 3D foot images from this system can reach 0.7 mm or higher. A 100% LED lighting environment also ensures the safety and comfort of the subjects during longer scanning sessions. System validation. Accuracy and repeatability are two important indicators of the robustness of measurements in measuring systems. According to ISO 5725-1:1994, accuracy is the degree of agreement between the test result and the true or accepted reference value. Precision is defined as the degree of agreement between test results obtained under the same stipulated conditions 38 . Good repeatability can ensure as few errors as possible  www.nature.com/scientificreports/ that are caused by measuring systems randomly. Therefore, a reliable measurement system should have both good repeatability and the ability to provide accurate measurements. In this study, the repeatability of the 3dMD foot scanning system is examined through repeated scanning of the same object under the same conditions. The accuracy is examined by comparing the difference in the measurements of the same object between the system and a market-available scanner with a relatively high degree of accuracy. To prevent unavoidable subjective errors caused by soft tissue deformation and body sway, a rigid foot model (see Fig. 2) and a rigid ball (ping pong ball) with a circumference of 40 mm are used to assess the measurement performance of the system in static and dynamic situations respectively.
Repeatability test. Repeatability is defined as the level of consistency between measurements of the same object taken under the same scanning conditions by using the system. The foot model and ball were therefore scanned three times by the same operator under the same scanning conditions to assess the repeatability of the system. The foot model was scanned with a self-weight bearing at 2 different views (the angle of the two views is 180°) respectively, see Fig. 3. The rigid ball was also scanned when rolling through the platform from two different directions (the angle of the two directions is 180°). The rolling speed is 1.4 m/s 39 , which is close to the average walking pace of humans. The system was switched off and on between each scan.  www.nature.com/scientificreports/ The raw scanned data of the foot model and ball were processed by using Geomagic Design X software to create meshes. These meshes were randomly selected and superimposed to calculate the mesh deviations of various pairs for repeatability assessments: (1) among the meshes scanned in the same view/direction, and (2) among the meshes scanned in a different view/direction. The accumulated areas (in % of the total mesh area) of the mesh deviations between each pair were calculated at 0.1 mm increments 40,41 . The mesh deviation is the difference between the scanned images created by the system. Accuracy test. Generally, accuracy is defined as the proximity of a measurement to its true or tolerant value. In this study, the 4D foot scanning system is compared with the EinScan Pro handheld 3D scanner (SHINING 3D®) with a foot station 1 , see Fig. 4. This scanner can process up to 3,000,000 points per second under the handheld scan mode. The handheld scanner shows a similar resolution and accuracy as the fixed scanner by using a minimum point distance setting of 0.2 mm. The accuracy of the scanned foot data with the use of this scanner is up to ± 0.5 mm. Both surface of the dorsal and plantar of the foot can be scanned simultaneously by using the foot station. A rigid foot model was scanned three times by the same operator who is well trained to use both the handheld scanner and 4D foot scanning system. Scanned images from both scanners were randomly selected and compared by using Geomagic, and the accumulated areas (in % of the total mesh area) of the mesh deviations were calculated at 0.1 mm increments between the selected pair. The mesh deviation is the measurement error between scans created by the two scanners. In addition, the circumference of scans of the ball from the 4D foot scanning system was measured 5 times to obtain the mean value for comparison with the original circumference of the ball so as to validate the accuracy of the system in a dynamic condition. 2) respectively. The dominant foot, which is determined to be the foot that kicks a ball habitually 42 , was scanned 3 times during stance with HWB and self-selected walking speed conditions respectively by using the 4D foot scanning system, see Fig. 5. The subjects have Type 1 or 2 diabetes mellitus (DM) in the early stages (self-reported with a diagnosis from a clinical physician) and the inclusion criteria 43,44 are those with no history of ulcers or neurological disorders (except neuropathy), and able to walk a length of 20 m repeatedly without a walking aid. Subjects who show the presence of active foot ulcers were excluded 45 . This study was reviewed and approved by the Human Subjects Ethics Sub-committee of The Hong Kong Polytechnic University (Reference Number: HSEARS20200128001). All methods were performed in accordance with the relevant guidelines and regulations. Information on the experimental requirements were provided to all of the participants and written informed consent was obtained from all the participants before the experiment commenced.

Foot anthropometric measurements.
Thirteen key foot anthropometric measurements were extracted for an indepth analysis of the foot deformation, including length: foot length (FL), medial ball length (MBL), lateral ball length (LBL); width: anatomical ball width (ABW), orthogonal ball width (OBW), and orthogonal heel width (OHW); height: ball height (BH), instep height (IH); girth: anatomical ball girth (ABG), and instep girth (IG); and angle: ball angle (BA), toe1 angle (T1A), and toe5 angle (T5A) 5,6 . According to the foot roll-over pattern defined by Blanc et al. 46 and Barisch-Fritz et al. 5 , five representative frames at first heel, MTH contact, and toe Point cloud registration and automatic foot measurements. Instead of processing the scanned images and taking foot measurements manually by using Geomagic, new approaches of point cloud reorientation for 3D foot image registration and automated measurement are proposed, to improve the efficiency of foot image analysis and measurement.
Point cloud filtering. Raw data obtained from 3D scanners is often corrupted with noise and outliers 47 . These errors can reduce the accuracy of the point cloud representation and post-processing, thus causing issues in determining the local point cloud features (e.g., normal vectors or curvature rate of change at sampled points), which can lead to incorrect values and then errors in point cloud alignment, data measurement, and other postprocessing tasks. Therefore, the first step of scanned image processing is 3D point cloud filtering, which is used to remove the noise and outliers 48,49 . A large number of variant filters have been proposed for different types of surfaces and noise models 48,49 , including the Statistical Outlier Removal filter 50,51 , Bilateral filter 52,53 , Radius Outlier Removal 54 , and Voxel Grid filter 55 , etc. However, none of these methods are appropriate as they require an iterative trial-and-error process to choose the algorithm and the parameters based on the scanned surface and noise characteristics. In this study, the noise and outliers of the foot images are removed by using Statistical Outlier Removal (Sor) filter before registration. This is carried out by calculating the neighbours of each point. The sparse outliers that do not meet the criteria are trimmed. For each point p i (i = 1,…, n) in the input dataset, the mean distance d i from p i to all its neighbours are calculated, see Eqs. (1) and (2). The result is assumed to be a Gaussian distribution with mean and standard deviations. The points that are not within a σ standard deviation from the mean distance are  www.nature.com/scientificreports/ defined as outliers and removed from the dataset 48 , which means if d i > d mean + 1*σ, then p i is removed. Figure 7 shows an example of data comparison before and after filtering.
Point cloud registration. After filtering, the 3D image should be re-aligned to a standard position based on local system to take the foot measurements 40,56 , see Fig. 8. First, the coordinate of the point set-first toe contact frame, was manually aligned to the standard position as the target point set. Then the other point sets were aligned to the target point set respectively by finding the points that correspond to each other. In this study, the alignment between the target and other point sets is regarded as a density estimation problem based on the CPD method. The target and other point sets are the two 3D point sets in this study: M J×3 = (m 1 , m 2 , …, m J ) T and S K×3 = (s 1 , s 2 , …, s K ) T , where J and K are the number of points in the two point sets respectively. The points in the target point set M were generated by using the GMM. The other point set S as the GMM centroids need to fit M by maximising the probability of GMM centroids S to generate M. This is to force S (GMM centroids) to move coherently as a group to preserve the topological structure of the point set 22 .
In this study, good approximation to the distance function between S and M is developed based on the CPD algorithm 57 . The problem of point set registration is regarded as distance function optimization, which is to minimize the errors to find the best space transformation between the point sets. An example of a 3D image registration result is shown in Fig. 9.
Automated extraction of foot measurement. Using image registration, all five selected frames were reoriented to a standard coordinate. According to the definition of each foot measurement 6 , the key anatomical points (P1-P10) were first extracted from each frame automatically. A total of thirteen foot measurements were taken, by using Eqs. (3) -(13) (Fig. 10). Also, the automated foot measurements based on 5 subjects were compared with manual measurements to examine the consistency of the two methods.
The key points (P1-P10) extracted from the foot images include: P1 and P10 which are the maximum and minimum Y values along the Y-axis. The foot outline projection in the X-Y plane (Fig. 11) consists of bumps extracted by using the Graham algorithm 58 . P2 is the point with the maximum absolute X value when the Y (1) d mean = n k=1 d k n   P8 and P9 are the points with the maximum and minimum X values along the X-axis with a 14-20% FL. The FL, MBL and LBL are the distance between the pternion P10 and P1, P4, P5 respectively along the X-axis.   www.nature.com/scientificreports/ The ABW is the distance between P4 and P5; OBW is the distance between P4 and P5 projected onto the X-axis and OHW is the distance between P8 and P9 projected onto the X-axis.
The BH and IH are the distance of P6 (61.8% of FL) and P7 (50% of FL) to the X-Y plane along the Z-axis, respectively.
The BA is the angle between lines P4P5 and P4P8 ; T1A is the angle between P2P4 and P4P8 ; and T5A is the angle between P3P5 and P5P9.
The ABG is the circumference that passes through P4 and P5 along the Z-axis, and the IG is the circumference that passes through P6 along the Z-axis. As shown in Eq. (14), Pi is the point at the cross sections (ABG, IG), and N is the number of points.
Data analysis. All of the foot measurements were analysed by using SPSS Statistics 21 software (IBM Corp., Armonk, NY, USA). A Shapiro-Wilk test was used to examine the normality of the 13 foot measurements. The results showed that all of the foot measurements are normally distributed (P > 0.05) in the different conditions. Intraclass correlation coefficients (ICCs) were used to determine the consistency of the automatic and manual measurement methods. A paired sample t-test was used to determine the significance between foot measurements during walking and in the stance condition with HWB. The significance level of the statistical analysis was set at 0.05.

Results
Repeatability of 3dMD foot scanning system. To evaluate the repeatability of the system, the 3D images of the foot model and ball were paired and selected randomly from 2 different scanning views/directions. Two randomly selected images were aligned using global and automatic alignment methods successively in Geomagic. Details of the pairs and results of the mesh deviations are listed in Table 1.

Selected pair
Percentage of overlapping area (Mesh deviation within 0.5 mm)

Percentage of overlapping area (Mesh deviation within 1 mm)
Static scanning (foot model) www.nature.com/scientificreports/ For the static scanning of the foot model, 98.83% of the areas of overlap between randomly selected images from View 1 has a deviation less than ± 0.5 mm, and 99.84% within ± 1 mm. Pairs in View 2 showed similar results; the overlapping areas between the meshes is 98.86% with a deviation less than ± 0.5 mm, and 99.80% within ± 1 mm. The overlapping areas of images randomly selected from Views 1 and 2 are up to 96.08% with a deviation less than ± 0.5 mm, and 99.56% within ± 1 mm.
For the dynamic scanning of the ball, 98.94% of the overlapping area between randomly selected images from Direction 1 has a mesh deviation less than ± 0.5 mm, and 99.95% within ± 1 mm. Pairs from Direction 2 showed similar results. The overlapping areas between the meshes is 98.46% with a deviation less than ± 0.5 mm, and 99.95% within ± 1 mm. The overlapping areas of images randomly selected from 2 different directions up to 98.92% with a deviation less than ± 0.5 mm, and 100% within ± 1 mm.
Accuracy of 3dMD foot scanning system. Meshes of the foot model from the 4D foot scanning system and EinScan Pro handheld 3D scanner were compared to examine the accuracy of the system under a static situation. It was found that 96.66% of the mesh overlapped between images randomly selected from the 2 scanners with a deviation less than ± 1 mm, see Fig. 12. For the accuracy validation of dynamic scanning, the circumference of ball scans randomly selected from 2 directions were measured 5 times by using Geomagic. The mean value of their measured circumference is over 99%, quite close to the original value of 40 mm.

Comparison of foot measurements between automatic and manual methods.
In this study, a total of 13 foot measurements of 5 subjects have been measured by using automatic and manual measurement methods respectively. ICC analysis was used to check the consistency of the foot measurements measured with the automatic and manual methods. In Table 2, the ICC value of each foot measurement is higher than 0.8 (P < 0.001), which indicates that the automated foot measurement shows a good agreement with the manual measurement.  female diabetic participants are listed in the Table 3, including age, weight, height, body mass index (BMI), foot size and years of diagnosis.
Comparison of foot measurements: dynamic vs. static conditions. The maximum (MaxDyn) and minimum (MinDyn) values of each foot measurement during dynamic walking were extracted to analyse the changes of each foot measurement during whole ground contact. In addition, the MinDyn and MaxDyn of the foot measurements were compared with those of HWB standing through a paired sample t-test respectively, see Table 4. As shown in Table 4, when the MaxDyn and MinDyn are compared, the measured foot values significantly increase from 1.2 to 28.4% in dynamic situations. From the first heel contact to MTH take off, the foot dimension changes are continuously measured and compared with MinDyn. Major increases in foot length and girth are found at the stance of first toe contact, while its dimension change in foot height is minimal. The highest increment in foot width is found at heel take off stance, see Fig. 13.
Then the MaxDyn and MinDyn are also compared with HWB standing respectively. The measured foot values significantly increase 0.5-35.7% from HWB to MaxDyn with the exception of orthogonal ball width and ball angle. While no significant differences are found with each foot measurement between MinDyn and HWB, except for ball width.

Discussion
Scanning performance of 3dMD foot scanning system. As shown in Table 1, the repeatability of randomly selected images from the same scanned views is similar (98% within ± 0.5 mm) in both static and dynamic scanning situations, which indicates that this system shows high repeatability from the same scanning views. The overlapping area from different static scanning views is slightly less than that from the same view, which might be attributed to the fact that View 2 is specialized for left foot scanning with the current equipment setting. The foot model is not a homogeneous object and therefore the repeatability of the scanned images in the two different views is slightly lower compared to that of the scanned images from the same view. For the homogeneous ball, the repeatability of images from 2 different directions does not have this problem. The repeatability of the  www.nature.com/scientificreports/ 4D foot scanning system excels that of 4D foot reconstruction system (4DFRS), which is a dynamic foot reconstruction system developed by Thabet et al. 59 , and has proven reconstruction differences that average 2.44 mm and 2.81 mm in static and dynamic situations respectively. In addition, Fig. 12 indicates that the accuracy of the system is similar as that of the handheld scanner with static scanning. However, meshes overlap less within a stricter deviation, but still meet the foot measurement tolerance in footwear manufacturing 60 . For dynamic scanning, the measurement of scans is well close to the original value. The accuracy of this system is higher than the dynamic foot scanning system proposed in Novak et al. 61 , the accuracy of which can achieve a scanning height and width better than 1.12 mm, scanning girth better than 1.73 mm, and scanning angle better than 2.41°.
Foot deformation at different stances while walking. In this study, compared to the foot shape at static situation, continuous changes of each foot measurement were investigated and found to have significant deformations at dynamic situation. For deformation of lengths (FL, MBL, and LBL): The dynamic lengths are larger compared to static stance with HWB. As shown in Table 4, the extension degree of MBL is larger than LBL. Toe allowance is crucial for shoes to enable a natural roll-over process 62 . Footwear manufacturer used to leave enough space in front of the toes for wearing comfort following the fashion trends based on subjective judgement. From this study, a tolerance of 3 mm in the lengths should be considered as dynamic allowance to optimise the fit of the toe box during walking.
For deformation of widths (ABW, OBW, and OHW): Compared to static stance with HWB, the widths reach the maximum value at heel take off stance. From this study, a tolerance of 5 mm in ball width should be considered for toe box shape design to accommodate the foot deformation. The heel width increased 4.8% during walking compared to static stance with HWB. Heel cup is recommended to enclose the heel soft tissue by providing additional confinement for the heel 63 .
For deformation of height dimensions (IH, BH): The arches collapse with increased loads upon the foot during walking 64,65 . Arch pads and metatarsal pads, which accommodate the morphological plantar deformation during locomotion can ensure the plantar fully contact the insole for better distribution of plantar pressure. As shown in Table 4, on the basis of arch pad thickness at HWB standing, a tolerance of 3-5 mm is further recommended for motion comfort.
For deformation of angles (BA, T1A, T5A): Besides toe allowance, the ball angle is also an important measure to determine the flex line location for shoes. In this study, it is interesting to find that the ball angle tends to be obtuse during walking. The angle between the flex and medial lines of the shoe for diabetics is thereby suggested to be approximately 90°. The forefoot seems more pointed since the toe angles increase during walking, especially at final push stances. The deformation of TA1 is slightly higher than TA5, which can be attributed to the fact that Hallux Valgus is commonly found in the feet of diabetic patients 66 . However, for shoe/insole design, pointed forefoot contour is not recommended due to toes spread out during standing. www.nature.com/scientificreports/ For deformation of girths (ABG, IG): The dynamic girth is larger than static stance with HWB, which might result from the contractions of intrinsic and extrinsic foot muscles 67 , as well as plantar fascia and ligaments deformation during walking 68 . As shown in Table 4, a 10 mm allowance should be considered to optimise the shoe girth fit during walking.
In this study, the continuous changes of foot shape were analysed from 5 representative stances, to investigate the dynamic allowance in design of diabetic footwear with optimal fit and comfort during walking. Generally, the findings from this study remedy the gap in the analysis of static foot deformation. Nevertheless, there are still some limitations in this study. First, the 5 frames which represent the whole ground contact are selected through visual observation, so manual errors are unavoidable even though this task is done by the same person. All of the frames of the foot roll-over process rather than a few of them are measured automatically for analysis. Therefore, the error caused by manual selection can be avoidable. Secondly, the number of diabetic subjects involved in this study is small (only female subjects), so the results cannot be generalized. More subjects can be recruited to analyse dynamic foot deformation in future studies.

Conclusion
This study has introduced an advanced 4D foot scanning system with good repeatability and accuracy for foot measurements in both standing and walking contexts. In comparison to traditional 3D scanning, the system can comprehensively and efficiently compare and visualise the changes in foot shape at different regions and during different stances of walking. In this study, the foot geometry changes and deformation of the feet of diabetic patients during their self-selected walking speed are continuously measured and compared by using the advanced scanning system. Compared with the foot measurements taken at static foot stance, foot length and girth dimensions at first toe contact and foot width at heel take off provide good references of foot shape changes during walking in design of diabetic footwear. Conventional footwear designs based on foot shapes taken with static stance in no/half weight bearing conditions inherently result in fit problems during walking, thus adversely affecting wear comfort and increasing the risk of foot pain and pressure. This study contributes by providing a practical recommendation for diabetic footwear and insole designs. Apart from adding toe allowance in designing the toe box and flex line, arch and metatarsal pads as well as a heel cup can be taken into consideration to reduce the risk for diabetic foot ulcers caused by excessive plantar pressure and deformation at dynamic situations. Skin-marker based motion capture is a frequently used method for foot movement analysis at different dynamic situations 69 . However, it is time-consuming and costly and detailed deformation of soft tissue cannot be captured 28 . Extended studies will be conducted to extract the information of foot motion functions from the new marker-less 4D scanner to give detailed recommendation on therapeutic insole design for diabetes.

Data availability
Data are available upon reasonable request. If you want to request the data from this study, please contact corresponding author, Dr. Kit-lun Yick.